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Purpose 
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• Bayesian reliability requires the development of a prior distribution to 
represent degree of belief about the value of a parameter (such as a 
component's failure rate) before system specific data become 
available from testing or operations. 

- Generic failure data are often provided in reliability databases as point 
estimates (mean or median) 

- A component's failure rate is considered a random variable where all 
possible values are represented by a probability distribution 

- The applicability of the generic data source is a significant source of 
uncertainty that affects the spread of the distribution. 


This presentation discusses heuristic guidelines for quantifying 
uncertainty due to generic data applicability when developing 
prior distributions mainly from reliability predictions. 
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• Uncertainty is represented as a probability distribution 
for a parameter, such the failure rate of a component 

• Understanding the sources of uncertainty, how to 
estimate it, and methods for reducing it support better 
decision making 

- Design and Development of Complex Launch Vehicles 

- Launch Readiness Decisions 

- Scenario and System Trade Studies 
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Background on Uncertainty 
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• Two types of uncertainty 

— Aleatory (inherent physical random variability) 

• Typically inherent in every system and cannot be reduced 

- Epistemic (lack of knowledge or ignorance) 

• Can be reduced by increasing knowledge 

• Epistemic uncertainty includes: 

- Completeness (missing scope/scenarios) 

- Parameter (component/subsystem) 

- Model (assumptions, system treatment) 

• This presentation will focus on epistemic uncertainty 
associated with the parameters of reliability models 
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Parameter Data Sources and the 
Concern of Applicability 





• New Launch vehicles (LV) comprise heritage and new 
hardware 

• Generic reliability data are collected from a wide 
spectrum of sources: 

- Component databases (NPRD, EPRD, NUCLARR, etc.) 

- Aerospace historical data 

— Other industry historical data 

- Piece part count method (MIL-HDBK-217F) 

- Engineering judgment 

• This variety of data sources raises the concern of 
source data applicability to target parameter 
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What is Data Applicability and why is N W 

it important 


• Applicability is the degree of credibility and relevance of the 
source data 


• Data applicability is a significant source of epistemic uncertainty 

• Bayesian methods permit the use of subjective information, and 
a heuristic approach is one way to consistently incorporate 
subjective judgments about applicability into the modeling 
process 

• The evaluation of applicability with a heuristic approach is most 
useful 

- When other Bayesian statistical methods are not available or yield poor 
results 

- To ensure consistent judgments throughout the model 








An approach for quantifying data 

applicability 





• Classify the applicability of the reliability data source 

• Assess the environmental conditions between the source and the target 
parameter 

• Apply the appropriate K factor (Logistic Performance Factors) to adjust 
the source data mean from its environment to predict the mean failure 
rate of the component in the application environment 

• For each data source, apply the heuristic guidelines to quantify the 
uncertainty due to applicability 

• Source applicability is divided into two categories 

- Data source application 

• Rank most applicable to least applicable data sources 

- Data source environment 

• Uncertainty increases when converting from one type of environment 
to another (i.e., GF to AUF, or AUF to AIF) 
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Data Source Classification Table for 
Heritage Hardware 





Source 

Category 

Source Descrption 

Source Application 

Source 
Application 
Error Factor 

Compatible 

Environment 

Adjusted 

Environment 


Specific Component Data 

different LV -same component 

3 




different LV- like component 

4 



Historical 

Aerospace Data 

Same LV component 

5 

No impact on 

Increases the 

Like component 

6 

Error Factor 

Error Factor 


Other Industry Data 

Same LV component 

6 




Like component 

7 




Note: The error factor (EF) is a measure of dispersion of the lognormal distribution. 

• EF = 95 th /Median = Median/5 th = sqrt(95 th /5 th ) 

• An EF of 1 is certainty (point estimate) 
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Data Source Classification Table for 

New Hardware 





Source 

Category 

Source Descrption 

Source Application 

Source 
Error Factor 

Compatible 

Environment 

Adjusted 

Environment 


PiecePart Method 

Same LV component 

8 



Prediction 

Like component 

9 

No impact on 
Error Factor 

Increases the 

Non-expert EngineeringJudgement 

Documented Process 

10 

Error Factor 


Documented/Undocumented Process 

15 
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Data Source Classification Approach N W 
The complete Table 


Source 

Category 

Source Descrption 

Source Application 

Source 
Application 
Error Factor 

Compatible 

Environment 

Adjusted 

Environment 


Specific Component Data 

different LV -same component 

3 




(Most Applicable) 

different LV - like component 

4 




Aerospace Data 

Same LV component 

5 




Like component 

6 





Other Industry Data 

Same LV component 

6 

No impact on 

Increases the 


Like component 

7 

Error Factor 

Error Factor 


PiecePart Method 

Same LV component 

8 



Prediction 

Like component 

9 





Non-expert Engineering Judgement 

Documented Process 

10 





(Least Applicable) 

Undocumented Process 

15 
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Case Study 
Simple Fault Tree 
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• A simple model consists of 4 components 1, 2 , 3 and 4 

• Component 1 is in parallel with components 2, 3, and 4 

• Components 2 , 3, and 4 are connected in a series configuration 


E ^ rbd_case_study OR RAM VM 2015 

! 0 C0MP0NENT1 [4.170E-007] Source: Non-Expert Judgement (Undocumented Process) 

- £ com p o n ents_i n_seri es AND 

1 0 C0MP0NENT2 (3.M-QQ7) Source: Piece Part Count 

! 0 COMPONENTS [1 ,390 E -007) Source: Aerospace Historical Data for same component 

i 0 COMPONENTS (4.865E-008) Source: Non-Expert Judgement (documented) 


Note: Numbers shown on this slide are examples only and do not represent data from NASA systems 



Case Study 

Uncertainty Quantification Results 

Runl 


IMA 


Uncertainty Results for Case 1 


5th: 4.73e-9 



5th: 2.86e-8 

| il l | |||||! | |||||||1|||| 


Mean: 1.58e-6 

95th: 6.35e-6 

1 



5th: 2.81 e-8 

111 


Mean: 1.34e-7 

95th: 5.15e-7 

1 HI 


Mean: 2.23e-7 

95th: 6.79e-7 



5th: 1.07e-7 


Mean: 1.87e-6 

95th: 6.9e-6 
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5th: 2.86e-8 

liiiwii: ::at^ 


Mean: 1.58e-6 

95th: 6.35e-6 


1e-9 


1e-8 


1e-7 


1e-6 


1e-5 


1e-4 


Probability (log scale) 

Model Error Factor (EF) = 95 th / Median = 16.2 


- Uncertainty results 



Component 4 (EF-10) 


Component 3 (EF-5) 


Component 2 (EF-8) 


Component 1 (EF-15) 





Case Study 

Uncertainty-Importance Analysis 




• The Uncertainty-importance routine identified component 1 as 
a major driver of the model uncertainty 

• A data research on Component 1 yielded more applicable data 

— Found historical data for a like component from the 
aerospace industry 

□■■A rbd_case_study OR RAM VM 2015 


\ o 

components jnjeries AND 



0 COMPONENT? (8, 354 E -007) Source: Piece Part Count 


■0 C0MP0NENT3 (L39QE-QQ7) Source: Aerospace Historical Data for same component 


0 C0MP0NENT4 (4.865E-008) Source: Non-Expert Judgement (documented) 
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Case Study 

Uncertainty Quantification Results 

Run2 


IMA 


Uncertainty Results for Case 2 


5th: 8.37e-8 


Mean: 6.8e-7 

95th: 2.16e-6 



5th: 2.83e-8 

™ sia^ 


Mean: 2.27e-7 

95th: 6.93e-7 



5th: 1.04e-7 


Mean: 1.81e-6 

95th: 6.63e-6 



5th: 8.37e-8 


Mean: 6.8e-7 

95th: 2.16e-6 
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1e-9 1e-8 1e-7 1e-6 1e-5 1e-4 


Probability (log scale) 

Model Error Factor (EF) = 95 th / Median 


= 5.6 


Uncertainty results 


Component 4 (EF-10) 


Component 3 (EF-5) 


Component 2 (EF-8) 


Component 1 (EF-5) 



Conclusion 


'NASA/ 






• Higher data applicability improves certainty of estimates 

• Uncertainty represents the spread of the parameter estimate. How 
confident are we that the estimate is correct 

- Useful for decision makers 

- Communicates credibility or lack of it 

• Adjustments for environments with k factors is another source of 
uncertainty 

• Highly applicable data in the model ensures lower uncertainty 

— Crucial step that increases the confidence level of the components and 
subsequent uncertainty estimate 

• Uncertainty-Importance routines can prioritize the need to collect 
additional parameter data 
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Questions? 


'NASA/ 



mm. 



POC: Mohammad AL Hassan (Mo) 
Mohammad. i.alhassan(g)nasa. gov 
205 - 544-2410 
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